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An analysis of a particular type of multiserver, multiqueue system is 
presented in which each queue has a finite number of waiting positions 
and the waiting positions are not vacated until service is completed. Thus, 
several customers in one queue can be served simultaneously. The steady- 
state distribution of states is derived and is used to obtain the probability 
of loss for each queue and the average delay of the system. This analysis is 
then used in the development of a design procedure to determine the 
minimum-cost configuration of waiting positions and servers to meet 
specified single-hour grade-of-service constraints. The results are applicable 
to the design of systems that utilize automatic call distributors. While this 
model does not include such effects as day-to-day variation and noncoinci- 
dence of peak loads among trunk groups, nevertheless the results properly 
reflect for the first time the interactions among the trunk groups terminated 
on the automatic call distributor and the attendants at the automatic call 
distributor. 

I. INTRODUCTION 

The purpose of this paper is to present an analysis of a particular 
type of multiserver, multiqueue system in which each queue has a 
finite number of waiting positions and the waiting positions are not 
vacated until service is completed. In particular, the steady-state dis- 
tribution of states are derived, and expressions for the probability of 
loss for each queue and the average delay are given. It is shown that 
the queuing model described is representative of systems characterized 
by a finite number of trunk groups that carry calls to a group of 
attendants who then perform some service for the caller. (One such 
system is used in the directory assistance service provided by the 
telephone company.) Results are given to illustrate the effects of vary- 
ing the number of servers and number of positions in each queue. 
Finally, a design procedure to determine the minimum-cost configura- 
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tions for such systems under various grade-of-service constraints is 
developed. This procedure ignores such effects as day-to-day varia- 
tion, noncoincidence of peak loads among incoming trunk groups, and 
retrials of blocked calls, which should be investigated in the develop- 
ment of procedures for traffic engineering and administration. However, 
as in most cases, it is difficult, if not impossible, to obtain analytical 
results with these effects included. This paper should provide useful 
insight that can later be incorporated in a complete traffic engineering 
procedure. 

The system analyzed consists of I input queues each with a finite 
number of waiting positions, Ni (i = 1, • • •, I), which have full access 
to M servers. When an arrival seizes one of the M servers, it does not 
vacate a waiting position, but remains in the position until its service 
has been completed. This characteristic, which allows calls that are 
not at the head of a queue to be in service, distinguishes this system 
from the usual queuing system. In particular, the system is no longer 
completely described by the number of calls in each queue since a 
record of the number of calls from each queue that are in service must 
be kept. An arrival that finds all the waiting positions for its queue 
occupied is cleared or lost from the system. An arrival finding no idle 
servers but at least one vacant waiting position in its queue enters the 
queue and is delayed until its service begins. In the context of directory 
assistance systems, the input queues are the trunk groups and the 
waiting positions are represented by the trunks. The information 
operators are the servers. 

In telephone traffic theory, the described system has been referred 
to as a combined loss-and-delay system. Previous work in this subject 
can be segmented into three parts : 



(%) One input flow — one queue. 
(it) Several input flows — one queue. 9-11 
(Hi) Several input flows — several queues. 



The last segment, of which this analysis is a part, has been investigated 
by Kiihn. He analyzed systems with g > 1 queues, each with a finite 
number of waiting positions s,- (i = 1, 2, • • •, g). Associated with each 
queue is a Poisson arrival process with mean rate X,-, which is assumed 
to be independent of the others. An arrival that finds all waiting 
positions in its queue occupied is lost. Arrivals that are not cleared from 
the system are served by one of n servers. The service time distribution 
for the ith server is exponential with a mean rate e,. When a server 
becomes idle, queue i is chosen to receive service with probability p,-. 
Within a queue, calls can be selected randomly, first-come, first-served, 
or according to a priority scheme. 
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As Kiihn indicates, analytical solutions to these systems exist only 
for special cases. In general, the linear equations representing the 
equilibrium conditions must be solved numerically for the particular 
values of the parameters. However, Kiihn gives a solution to one 
particular system, which will now be discussed. In this system, it is 
assumed that the service rate for all servers is identical (e,- = e) and 
the interqueue discipline is defined by the p/s that are 

Pi = g ' U = li ■■-! g), 

where Z, is the number of waiting positions occupied in the jth queue. 

The system analyzed in this paper is an extension of the one ex- 
amined by Kiihn, since an arrival does not release a waiting position 
until his service has been completed. This complicates the state 
analysis, since it is now not sufficient to know the number of servers 
that are busy to determine the equilibrium equations; information as 
to the number of calls from each queue that are in service must be 
included. 

Kiihn indicates also that, for the above interqueue discipline, the 
waiting time distribution can be found numerically only for small 
systems. The calculation of the waiting time is complicated by the 
fact that an arrival's waiting time is influenced by the number of 
arrivals that occur after it has entered the system. More is said about 
this difficulty later. 

II. MATHEMATICAL FORMULATION 

In this section, a mathematical model of the queuing system is 
formulated. Equilibrium equations are given and their solutions 
derived. 

2.1 Queuing model 

The queuing system consists of I input queues, each with a finite 
length denoted by Ni, i = 1, 2, • • ■, I. Requests for service arrive at 
queue i according to a Poisson distribution with mean rate, X,. If we 
let Ai(t) denote the number of arrivals at queue i in (0, t), then 

P£ Ai (t) = fc] - M! e-ut (fc = 0, 1, 2, ■ • •)■ (1) 

The arrival process at queue i is assumed to be independent of the 
arrival process at each of the other queues. Arrivals from each queue 
have full access to a group of M servers. The service time distributions 
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of the servers, denoted by H(t), are independent and identical ex- 
ponential distributions with mean service rate /t, i.e., 

»h;~" s<i (2) 

Since the arrival process is Poisson and the service process is ex- 
ponential, the queuing model is a multidimensional birth-death process. 

Arrivals to queue i that find Ni waiting positions in queue i occupied 
are not allowed to enter the system. If an arrival to queue i finds at 
least one unoccupied position in queue * but all M servers busy, it 
enters the queue and waits as long as necessary for service. Within 
queue i, arrivals enter service on a first-in, first-out (fifo) basis. An 
arrival that finds at least one unoccupied position in its queue and at 
least one free server immediately enters service. When an arrival 
enters service, it does not release a waiting position but remains in 
the queue until its service has been completed. Hence, the word 
"queue" is being used in a nonstandard manner and refers to the 
number of calls waiting for service and in service. As discussed earlier, 
an example of such a queue is a group of trunks that carry calls into 
a switchboard. 

The interqueue service discipline — the order in which the queues 
receive service — is characterized by the number of calls waiting for 
service. When a server becomes free, queue i receives service with a 
probability, p., which is the proportion of queue-i calls waiting for 
service. If we denote the number of calls in queue i by m and the 
number of calls in queue i that are in service by m,-, then this prob- 
ability, which is dependent on (wi, n 2 , • • •, n h mi, • • -, mi) = (n, m), 
can be expressed as 

. . m — mi rii — m t ( i, ^ ... , - \ 

V^ /.. ... \ V- ,, HIT \J~L / 



£ (rij — m 3 ) X) % — M 

3=1 y=i 



(i= 1,2, ••-,/). (3) 



The effects of other interqueue service disciplines have been in- 
vestigated, but will not be discussed here. 

The fact that arrivals remain in the queue during service dis- 
tinguishes this queuing system from the standard system, since the 
amount of information required to fully describe a state of the system 
is increased. The system is also complicated by the fact that the inter- 
queue service discipline is state-dependent. However, as is shown in 
later sections, this "complication" leads to a closed-form solution of the 
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equilibrium equations, which is generally not the case for such systems. 
The equations of equilibrium are given in the following section. 

2.2 Equilibrium equations 

The system described in the previous section is characterized by a 
finite number of states that indicate the number of calls in each queue 
and, of these calls, the number receiving service. We denote by (m, n%, 
• • • , ni, mi, • • • , mi) the state in which there are n, calls in queue i 
with to,- of these calls in service. For notational simplification, we also 
refer to the state in vector notation as (n, m). In this notation, (n,-+, m) 
represents the state (ni, n 2 , ••-,»,-+ 1, ■ * •, »i, wii, • • ■, mi) and 
similarly (n,_, m) = (n„ •••,?!,— 1, • • •, mi). It should be clear that, 
if the total number of calls in the system is less than or equal to M, 
then ni = mi for all i. 

Assuming stationarity, let P(n, m) be the probability that at an 
arbitrary instant of time the system is in state (n, m) . Moreover, if the 
arrival processes to the system are Poisson, the equilibrium-state 
distribution {P(n, m) } at an arbitrary instant is equal to the equilib- 
rium-state distribution at the instant of an arrival. By equating the 
rate into a state to the rate out of a state, we can write the equilibrium- 
state equations where we have introduced the function 

to include the boundary conditions and a, = X,/w : 
£ LdiuiNi - m) + n,]| P(n, m) 

{ 2 

= £ aiU(ni)P(ni-, m^) + £ (n,- -f l)u(Ni - n,)P(n, + , m l+ ) 
»-i i=i 

(Zni<J\A (O^riigNi) i=l,2, ■■■, I (4) 

£ [aiu(Ni - n,i) + m,] P(n, m) 

i i 

= £ aiU(n,)P(n,_, m,_) + £ mMNi - n,)P(n i+ , m) 
t-i t— i 

i i 
+ £ E (wi + l)u(Ni - n,)M(TOy)P(n t+ , m,- +J _) 

■ = i j ■ = i 

irH 
( .E ni = M\ (0 g m = Ni) i = 1, 2, • • ., I (5) 
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E ZaMNi - m) + mi]} P(n, m) = E a^Cn^n,-, m) 



w(A^i - n,)P(n,+, m) 

w(ATi — n,)u(m>)P(n, + , m, +J _) 



(0 ^ n,- g Nt) i = 1, 2, 



(6) 



, J, m f (wj + 1 — m,j) 

1=1 ( £>*+! " M ) 

< « (m,+ l)(n,+ l -my) 
T 2- 2- T - j \~~ 

(i iN>jf ) 

Equations (4) to (6), together with the normalization condition 

Ni Ni min (M,N,) min (M,Ni) 

L • • • £ E • • • E ^(n, m) = 1, (7) 

ni =0 ni=0 »u =0 mi =0 

where all nonexistent states, such as states in which both n, = and 
mi > 0, in the sum are assumed to have probability zero, determine the 
equilibrium-state distribution. 

2.3 Steady-state solution 

Since the process described by the equilibrium equations is a 
finite-state birth-death process in which the arrival rate into the 
system is always less than the service rate of the system as a result of 
overflow from the finite queue, a unique solution to eqs. (4) to (7) 
exists, and the solution is a genuine probability distribution. 16 This 
solution is given in terms of P(0, 0) by 



^(n, m) = 



a? 



fl^mo) 



{&«*") 



(8) 



En,-- M 
ni — mi, • • • , ni — mi 



M 
[mi, m 2 , • • • , mzj 



t-1 



M ! M lni ~ M 



P(0, 0) 



(x>,>m), (9) 



where P(0, 0) is determined from (7). The general solution was de- 
termined from examination of various small systems. By substitution, 
it can be shown that this solution in fact satisfies the equilibrium- 
state equations (4) to (6) . 

When the number of calls in the system is less than or equal to the 
number of servers, no one is waiting for service and the queues have no 
interaction. This fact is shown in (8) by the product form of the solu- 
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tion. However, as the number of calls increases to the point where calls 
are waiting in more than one queue, the queues no longer are acting 
independently. 

Since the queues behave independently as long as there are free 
servers, we would expect that, as the number of servers was increased, 
the system would approach I independent loss systems. By examining 
the marginal probabilities, it can be shown that this is, in fact, true 
when M = E'=i Ni. That is, the marginal state probability of n x 
calls in queue 1 is 

ar/ri! ! 



P(ni) = E 



E P(n, m) = K 

E af/fc! 



(10) 



k=i 



which is identical to the state probability for a pure loss system. 

If no calls were blocked from the system, then the system would act 
as a pure delay system with the offered load a — Et=i <**• This is 
easily shown by taking the limit of (8) and (9) as JV\- — * « for all 
i=l,2, ■■■, I. 

We first consider the case in which E'=i n% ^ M. Since the number 
of calls in each queue is unrestricted, it is easily seen that the multi- 
nominal expansion of (cti + ■ • ■ + ai) 2n ' divided by (E*=i n d 1 can be 
obtained from (8). That is, 



lim £ P /„ _x (oi + 

i=l,2,- •-,! 



+ 0|)' 



k\ 



P(0, 0) 



(in<<M) 



Hence, 



P ( ,£ * - fc) = £ { P(0, 0) (k < M). (11) 



For the case in which E'=i »i = M, first note that, by the Vander- 
monde convolution of multinomial coefficients, 



E n>i 



M 



ni — mi, • • ■ , ni — mi 



Therefore, 



P(n) = E P(n, m) = 



(£-)■ 



M 



m h • • • , wj 



wi, n 2 , • • • , ni 



M ! Af 2 --" £ft m ! 



n ^s p(o, o) 



(12) 



(l^^), (13) 
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P(0) = 



and thus we can denote this state probability by 

Sn,=A: 

i-l 

(k ^ M). (14) 
Consequently, 

Jim P ( £ m = fc) = MI m^ P(0 > 0) (/c = M) - (15) 

The normalization constant is then expressed as 

Hence, comparison of (11), (15), and (16) with the pure delay system 
completes the proof. 

2.4 Blocking and delay probabilities 

In the analysis and design of queuing systems, performance mea- 
sures for each configuration must be calculated. In telephone traffic 
theory, these performance measures are generally referred to as 
"grades of service." Two such measures of the grade of service are: 

(i) For loss systems, the blocking probability or probability of loss. 

(it) For delay systems, the average delay experienced by calls that 
enter the system. The average delay W(s, a) for a pure delay 
system is expressed in terms of the Erlang delay formula as 

*<•• «> - ^&- (17) 

In the system described in Section 2.1, the blocking probabilities 
for each input queue, the average delay experienced by calls that enter 
the system, and the average delay of only those customers who experi- 
ence a positive delay are important characteristics to be examined. The 
latter is not used in the remaining analysis. 

The blocking probability for queue i is defined as the probability 
that an arrival to queue i finds Ni calls in the queue. This probability, 
which is denoted by 2?.(N, M, a), is a function not only of the number 
of positions in queue i and the offered load to the queue, but also of 
the traffic load offered to each of the other queues, the number of 
positions in each of these queues, and the number of servers. Recalling 
that, for systems with Poisson input, the state probabilities at an 
arbitrary instant are equal to the state probabilities at arrival times, 

602 THE BELL SYSTEM TECHNICAL JOURNAL, MARCH 1975 



-B,(N, M, a) is calculated from the marginal distribution for queue 
i as 

5,(N, M, a) 

= £••• £ £ ■•■£••• E P(m, • • ', Nt, • ■ ', n h m). (18) 

ni n,-i n,+i mi ml 

The average delay experienced by calls that enter the system (suc- 
cessfully occupy a waiting position in their queue) is denoted by 
D(N, M, a). The delay calculated for this system is the overall mean 
waiting time measured from an arrival's entry into its queue until its 
service begins. Hence, it does not include service time of the call. It 
should also be noted that arrivals into the system that find at least one 
free server experience no delay. 

Since the average number of calls waiting for service must be finite 
and the mean waiting time is finite as a result of the loss structure of 
the system, the well-known equation of Little, 16 L = \W, can be used 
to calculate D(N, M, a). In particular, we must define our "queue 
length" as the total number of calls waiting for service and "X" is 
defined as the effective arrival rate into the system. Hence, 

£ ■ • • £ £ • • • £ |Y E n, - m) P(n, m) 1 

E n { >M 

D(N, M, a) = -^— (19) 

£ \£1 - BiiN, M, a)] 
i =i 

Calls that are blocked from the system do not enter the queue and 
hence do not affect the average queue length. Consequently, they are 
not included in the arrival rate into the system. The numerator of (19) 
is the average number of waiting calls. It should be apparent that the 
average delay for any particular queue can easily be obtained by using 
the appropriate marginal probabilities. Also, the conditional average 
delay, the average delay experienced by only those that must wait, 
is found by dividing (19) by the probability of being delayed. 

For some design purposes, it might be deemed necessary to constrain 
the probability of waiting longer than some time, t , to be less than a 
specified value. In this case, the waiting time distribution for each 
queue must be obtained. For the interqueue discipline examined for 
this system, the calculation of the waiting-time distribution is ex- 
tremely difficult. (Kiihn 13 mentions that, for his problem, numerical 
techniques can be used for very small systems, after which approxi- 
mate methods must be formulated.) The difficulty in determining the 
waiting-time distribution lies in the fact that the time a particular call 
must wait for service is not just a function of the number of calls in 
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the system when it arrives, as is usually the case. In particular, the 
waiting time of a call is related to the number of calls that arrive after 
the particular call enters the system, since queues are chosen for 
service according to their queue lengths at a service completion, so 
that later calls can "pass" earlier ones. For this reason, the waiting 
time distributions are not calculated in this study. 

2.5 Macrostate analysis 

As discussed in the previous section, blocking probabilities and 
average delay values are of interest to system designers. Using the 
state probabilities given in (8) and (9), it is possible to obtain not only 
the overall average delay, D, as shown in (19), but also the average 
delay, Di, for queue i. In certain types of design, we might want to 
engineer the system so that the average delay in every queue is less 
than a specified level. In such cases, D t would be needed. However, for 
this study, we consider only the overall average delay. 

Therefore, it is apparent from (19) that, for computational purposes, 
we only need to know the number of calls in each queue without dis- 
tinguishing between those in service and those waiting. If we denote 
by n the state (m, n 2 , • • •, ni), we can find the steady-state prob- 
abilities P(n) from (8) and (9). Of course, we could have written the 
state equations directly and solved this easier set of equations. 17 
However, for further studies, it is essential to know the probabilities 

P(n, m). 

Since the state probabilities for (n, m) in which £t=i n* ^ M are 
independent of m, P(n) = P(n, m). To obtain P(n) for XUi n, > M, 
we sum P(n, m) given by (9) over all possible values of m. Using the 
Vandermonde multinomial convolution, we find that 

(k nt ) 1 • a- 

P(n) = £ • • • £ P(n, m) = ^J. M U £i P(0), (20) 

where P(0) is the normalization constant. We can calculate the block- 
ing probabilities and the average delay as before. The number of 
states is IIUi (#,- + 1). 

It should be repeated that the macrostate probabilities are of use 
only if one is interested in the overall mean delay. To calculate the 
individual average delays, one must use the microstate probabilities. 

III. RESULTS 

In this section, we investigate the effects of varying iV, and M on 
the blocking probabilities for each queue and the overall average delay 
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of a call. In particular, these effects are illustrated by comparing the 
results obtained from the analysis presented in this paper, for a partic- 
ular example, with the results obtained if independence is assumed. 
This "independence assumption" is often used in practice to deter- 
mine the number of positions for each queue and the number of 
servers needed. In essence, this assumption permits a designer to 
design each queue (trunk group) independently of the other queues 
and the number of servers, and the number of servers is determined 
assuming that no calls are blocked in the queues. It is shown that this 
assumption is generally not even a good approximation. Finally, four 
properties that are used in a design procedure established in the next 
section are postulated. 

3.1 Comparison of results with Independence assumption 

In the engineering of automatic call distributor systems, a traffic 
engineer generally dimensions each trunk group using the Erlang loss 
formula (assuming that it is independent of the other groups and the 
number of attendants) and often determines the number of attendants 
required from the Erlang delay formulas using the total offered load 
to the queues (assuming no blocking in the queues). This procedure is 
clearly invalid, but up to now an exact procedure has not been avail- 
able. As a means of illustrating the significance of the results presented 
in this paper, we now compare, for a particular system configuration, 
the system characteristics that a traffic engineer would expect to 
obtain using the independence assumption and what he really will 
find. Of course, the interqueue service discipline will affect these 
results in a way that will be described in later work. The actual opera- 
tion of such systems is quite complicated, and is not readily character- 
ized by any of the disciplines usually used such as first-in, first-out, 
random, and last-in, first-out. However, the results of simulations 
indicate that the discipline presented here is a good approximation of 
the actual method of operation. 

For purposes of this comparison, we assume a simple system with only 
two queues : the first with an offered busy-hour load of 10 erlangs ; the 
second, 5 erlangs. It is further assumed that the queues have coincident 
busy hours and that the average holding time per call is 30 seconds. 
Assuming that a P. 01 grade-of-service constraint has been placed on 
each group, the number of trunks required, if independence is assumed, 
would be Ni = 18 and N 2 = 11. (These numbers can be found from 
tables of the Erlang B formula.) If it is then required that, on the 
average, no call must wait longer than 3 seconds for an answer, we 
find, from the Erlang delay formula, that M = 19 (assuming no 
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blocking in the groups). The traffic engineer would expect this system 
to have the following characteristics : 

Blocking probability for group 1 = 0.0071. 
Blocking probability for group 2 = 0.0083. 

Overall average delay = 1.83 seconds. 

However, analyzing this system configuration with the results of this 
paper, we find that the service levels (which stated above are approxi- 
mations to the actual levels) would be 

Blocking probability for group 1 = 0.014. 
Blocking probability for group 2 = 0.013. 

Overall average delay = 0.949 second. 

The directions of the changes are intuitively obvious, since longer 
holding times in the queues result in more blocked calls and the higher 
blocking levels decrease the load to the servers, which in turn results 
in a lower average delay. It should be noted that the trunk groups 
are performing at unsatisfactory levels, but the overall average delay 
has been decreased and is considerably under the required level. Often, 
customers who have such systems measure only the delay or speed 
of answer and periodically remove attendants if they feel that the speed 
of answer is not above the required level. Unfortunately, such a 
customer generally does not realize the effect of removing attendants 
on the blocking probabilities on his incoming trunks and consequently 
on other network customers. 

As an example of customer behavior, consider the system discussed 
above. The customer, having measured the average delay and finding 
it to be considerably under his required level, would most likely re- 
move two attendants. The average delay would then become 3.01 
seconds, but the blocking probabilities increase to 0.028 for group 1 
and 0.023 for group 2. Hence, even though the delay constraint is 
essentially satisfied, the probabilities of loss are more than double their 
desired levels. 

If, instead of the P.01 service level, P.05 or P. 10 had been chosen 
for the above delay constraint, the independence assumption would 
generally give a configuration that would satisfy all service con- 
straints. The reason for this is that the higher blocking levels decrease 
the offered load to the attendants, and consequently a very small delay 
results. This delay is small enough that it has little effect on the hold- 
ing time of the calls and, hence, the offered load to queue i is ap- 
proximately a,. Therefore, the resulting blocking probability, although 
larger than the Erlang loss probability, is generally in the acceptable 
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range. However, the configuration would not be optimal because too 
many attendants are provided. In summary, as the blocking prob- 
abilities increase, the discrepancies between the Erlang loss formula and 
the formula given by (18) decrease, but the discrepancies between 
average delays increase. 

Several variations in the procedures to engineer these systems exist 
in practice. Generally, in determining the number of attendants, the 
measured offered load into the attendants is used. This load accounts 
for the blocking in each trunk group. For the above example, the 
measured offered load would be 14.79 erlangs (if we assume that the 
assumptions made in this analysis are valid) and, from the Erlang 
delay formula, an average delay of 1.58 seconds results, which is still 
higher than the actual delay. The reason for the discrepancies is that 
the offered load to the attendants is no longer Poisson as a result of 
the blocking in the groups. In fact, the variance of this offered load will 
be lower than that of the Poisson load, since the peakedness of the 
traffic has been decreased by clipping. Hence, since the actual average 
offered load and variance of the load are lower, this leads us to postulate 
the following property : 



Property 1 : 



W(M, a) ^ D(N, M, a) where a = £ a,. 



This property is illustrated in Fig. 1. The equality holds in the limit 
as Nt — >°° for all i = 1, ■ • •, I, as shown previously. The significance 
of this property is that we now have a method of obtaining an upper 
bound on the average delay for the combined system. 

Another variation that is sometimes used is to add the speed of 
answer into the offered load to each group. If we add the average 
delay of 0.949 second to each call and use this new offered load in the 
Erlang loss formula, the blocking probabilities that result are 0.0091 
and 0.01, for groups 1 and 2, which are still lower than the actual 
blocking. 

The discrepancies result because the Erlang loss formula assumes 
exponential holding times on the trunks but, in fact, the holding time 
for the combined system is the sum of an exponential distribution and 
the delay distribution. Also, the holding times of calls in the system 
are no longer independent (unless 5Z' (= i n< < M). The variance of this 
new service time distribution is higher than that of the exponential 
service time distribution and, of course, the mean is larger. Therefore, 
one would expect the average queue length to be larger which, in 
turn, implies an increase in blocking. 
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Fig. 1 — Average delay as a function of number of positions in one queue. 

With these facts in mind and as a result of empirical evidence, we 
postulate a second useful property : 

Property 2: 

B(Ni,ad £BiQK,M,a) (t = 1, ••-,*), 
where B(Ni, a,) is the Erlang loss formula for Ni servers and an offered 
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Fig. 2 — Blocking probability as a function of number of positions in the queue. 

load, a,. The equality holds when M ^ SAT,, since the queues then 
behave independently (there is no delay, so in fact the holding time 
per call is exponential). This property is illustrated in Fig. 2. In- 
tuitively, one would expect B,(N, M, a) to be larger since, as a result 
of a positive delay added to each call, calls hold the trunks longer, 
therefore increasing the probability of an arriving call finding all 
trunks busy. The significance of Property 2 is that a lower bound on 
the number of trunks required for a given service level can be found 
using the Erlang loss formula. 
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3.2 Effects of varying W ( and M 

For a given input process and service time distribution, the designer 
can affect the blocking probabilities for a queue or the average delay 
by changing the number of positions, iV,, in a queue and/or changing 
the number of servers, M . We first investigate the effects of varying 
the number of servers, M. As noted earlier, when M has been increased 
to the point where ^' =1 AT» = M, the average delay becomes zero 
and the queues behave independently. The blocking probability for 
each queue is then given by B(Ni, a*) which, by Property 2, is a lower 
bound on the blocking for any value of M. What is of importance, 
however, is: Do the blocking probabilities and the average delay 
monotonically decrease to their lower bounds as we increase M to the 
value Z)Ui Ni? Empirical evidence, such as shown in Figs. 3 and 4, 
indicates that the answer to this question is yes. We postulate this 
property as : 

Property 3: 

Bi(B, M + 1, a) ^ 5<(N, M, a) (i = 1, • • •, I) 
D(N, M + 1, a) ^ D(N, M, a). 

Intuitively, one would not expect that increasing the number of 
servers in a system would increase the average delay. Moreover, a 
decrease in the holding time of calls would imply that the average queue 
lengths would decrease, which would result in a decrease in the blocking 
probability for that queue. However, this decrease in blocking results 
in an increase in offered load to the servers but, as we postulate, this 
increase is less than the marginal carrying capacity of the added server. 
The significance of this property is that, with added servers, not only 
is the average delay decreased but also the blocking probabilities are 
decreased ; that is, adding servers improves the service performance of 
the servers and of the queues (trunk groups). 

The other system parameters that may be varied to improve system 
performance are the numbers of positions in each queue. Supported by 
quantitative evidence, such as given in Figs. 1 and 5, and by intuition 
we postulate the following : 

Property 4' 

Bi(N i+ , M, a) g BiCN, M, a) 

5 y (N i+ , M, a) ^ 5,(N, M, a) j * i. 

D(N i+ , M, a) ^ £(N, M, a) 

The first part of this property states that, if we increase the number of 
positions for calls to occupy in a given queue, then the probability of 
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Fig. 3 — Average delay as a function of number of servers. 

loss for that queue is decreased. (This, of course, is true in pure loss 
systems.) The intuitive argument is that calls that previously found 
Ni calls in queue i were blocked, but now are not. Therefore, the 
number of calls blocked is decreased. However, as the third part of 
this property implies, the average delay of calls is increased as a 
result of this increase in calls from queue i. The intuitive counter- 
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Fig. 4 — Blocking probability as a function of number of servers. 

argument is that this increase in holding time per call might result in 
an increase in blocking for queue i. But we postulate that the increase 
in delay is not substantial enough to eliminate the increased efficiency 
obtained in queue i by the addition of a position. 

However, for the other queues, the number of positions remains 
fixed, and this increase in average delay results in a larger traffic level 
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Fig. 5" — Blocking probability in one queue as a function of number of positions in 
another. 



per position being placed on the queues. Hence, this increase in load 
causes the average queue length to increase and, consequently, causes 
the blocking probabilities to increase. This fact is stated in the second 
part of Property 4 and is illustrated by the example given in Fig. 5. 
Equality in the three statements holds only in the limit as iV, goes to 
oo for all i — 1, 2, • • •, I. 

The significance of Property 4 is that the loss probability for a 
given queue cannot be reduced by adding positions to any other queue. 
Therefore, by Properties 3 and 4, we see that the loss probability for a 
given queue can be reduced only by increasing the number of servers 
or by increasing the number of positions in that queue. 

The four properties postulated indicate relationships between the 
system parameters and the system characteristics. Proofs for the 
simplest cases (i.e., I = 1 for all the properties except for the second 
part of Property 4 for which I = 2) are given in Ref. 18. The proofs 
for the general cases have not been constructed because of the dif- 
ficulties involved (e.g., the proof of the second part of Property 4 
required 17 pages for 1 = 2). However, based on the intuitive explana- 
tions given, empirical evidence, and these proofs, I feel that the 
properties are valid in the general case of I queues. To obtain an optimal 
configuration (minimum cost), one must balance the cost of servers 
against the cost of positions for the queues in such a way that all 
required service levels are met. These properties are used in the next 
section in the development of a procedure to determine this optimal 
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configuration. The algorithm and proof of convergence given below 
assume the validity of these properties. 

IV. SYSTEM DESIGN WITH GRADE-OF-SERVICE CONSTRAINTS 
4.1 The design problem 

In determining the optimal design of a queuing system, one is 
generally interested in minimizing the operating costs in such a way 
that specified grade-of-service constraints are met. These constraints 
are a function of the particular queuing system under study. In the 
pure loss system, the grade of service is measured by the probability that 
a call is lost or blocked. Hence, the optimal system configuration is the 
minimum number of servers that satisfies the constraint, B(s, a) ^ 6. 

In delay systems, at least three measures of service are useful. The 
first is the average delay experienced by a call in obtaining service. 
The second measure is the "extremal" delay — the probability that the 
delay for any call exceeds a specified limit. Finally, the average delay 
of only those customers who experience some delay is a useful measure. 

However, in the combined loss and delay systems described in Sec- 
tion II, the determination of an optimal configuration is not as straight- 
forward. We will measure the grade of service of the system by 

(t) The blocking probability for each queue. 
(it) The average delay of all calls that enter the system. 

The blocking probability for a particular queue is dependent on the 
number of positions in each queue and the number of servers, and the 
average delay depends on these same variables. A procedure must be 
developed to balance these measures of congestion in such a manner 
that the costs of the system are minimized. 

More formally, the problem can be expressed as the following 
nonlinear problem in integer programming. We denote the monthly 
cost of a waiting position in queue i by C, and the monthly cost of 
each server by C. It is assumed that d and C are positive, finite 
numbers. The blocking objective for queue i will be denoted by &< and 
the average delay objective by d. The following assumptions have been 
made : The system is engineered for the system busy-hour traffic load 
and the busy hours for the queues are coincident. The optimization 
problem is then expressed as : 



Minimize the cost 



Z = £ CiNi + CM 

1=1 
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subject to 








(I) 


Bi(N, M, a) g 6, 


(i= 1,2, ...,I) 


(21) 




5(N, M, a) S d 




(22) 




Ni(i= 1,2, .-.,0, 


Af ^ integers. 





Since no efficient algorithm to solve a general nonlinear integer- 
programming problem exists, a procedure was developed that utilizes 
the four properties stated in Section III. 

4.2 Optimization procedure 

In this section, a procedure is developed that determines an optimal 
solution to the nonlinear integer-programming problem expressed by 
(I). The procedure is a direct-search routine based primarily on the 
properties presented in the previous section. A description of the 
procedure is now given and is followed by a concise summary of the 
algorithm. Figure 6 is a flowchart of the algorithm. 

The first step in the algorithm, as in most mathematical program- 
ming algorithms, is the determination of a feasible solution to the 
problem. To obtain an initial feasible solution to (I), we utilize Proper- 
ties 2 and 3 of the previous section. In particular, by Property 2, we 
know that the minimum number of waiting positions for queue i can 
be determined from the Erlang loss formula, which is easily computed 
from a recurrence relationship. 19 We begin the search for an initial 
feasible solution with 

Nf* = min [n\B(n, a,) ^6,}, 

since it has been shown that, in fact, a feasible solution to (I) exists. 
That is, (N {0) , M), where M = £'=i Nl 0) , is a feasible solution since 
£(N<°>, M, a) = and 5,(N<°\ M, a) = B(Nl 0) , a,) for i = 1, 2, • • -, I. 
However, since this solution will generally not be near the optimal 
solution, the search will not begin at M but instead with M (0 ), which 
is the minimum value of M that satisfies the constraint : 

W(M, a) ^ d. (23) 

M (0) is the number of servers that would be selected if the Erlang 
delay formula with an offered load a = Zi=i Ot were used. By Property 
1, it is seen that if (23) is satisfied, then (22) will also be satisfied. 
Using the set of parameters (N (0) , M w ), the system characteristics, 
{5i(N (0) , M ( o), a)} and 2)(N (0) , M (0)) a) are determined. One of two 
results occurs : 
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(i) The parameters (N (0) , M(o>) satisfy the constraints of (I), in 
which case an initial feasible solution has been determined. We then 
proceed to find the feasible solution of minimum cost. 

(ii) At least one of the blocking constraints (21) is violated. (As 
noted above, the delay constraint will be satisfied.) By Property 3, 
we know that the addition of a server will reduce the blocking prob- 
ability for each queue and that, by the addition of enough servers, a 
feasible solution can be obtained. We denote this initial feasible solu- 
tion by (N (Q) , M (0) ) and note two interesting properties of this solution. 

(a) By Property 2, NjP is the minimum number of positions that 
must be considered for queue i. 

(b) M {0) is the maximum number of servers that must be con- 
sidered. This is true since a further increase in the number of 
servers can be justified only if some waiting positions can be 
eliminated, and the positions are already at their minimum 
levels, N<°>. 

The next step in the algorithm is to attempt to improve the initial 
feasible solution. Since by construction we are initially at the maxi- 
mum number of servers, we attempt to decrease the costs by decreas- 
ing the number of servers while maintaining feasiblity. To maintain 
feasibility, it may be necessary to add waiting positions to certain 
queues. If a feasible solution is found, then it will be an improvement 
only if the accumulated cost of those servers removed (since the last 
feasible solution) is greater than the accumulated cost of all waiting 
positions that have been added to maintain feasibility. Hence, as we 
remove servers, one of three things results. 

(i) All the constraints of (I) are satisfied. If the accumulated cost 
of removed servers is greater than the cost of all waiting positions that 
have been added, this new feasible solution represents a cost improve- 
ment and should be stored as the tentative "optimal" solution. The 
accumulated costs are set to zero, a server is removed, and the search 
continues. If the cost of servers is less than the cost of positions, then 
we reduce the number of servers by one and continue the search for a 
solution with lower cost. 

(it) The delay constraint (22) is violated. In this case, we stop the 
search and the tentative optimal solution is the global* optimum. (A 
justification for stopping the procedure is given later.) 

(Hi) At least one of the blocking constraints is violated. In this case, 
we add one position to each queue in which the corresponding block- 



* I have taken the liberty of using "global" since, in fact, the procedure does 
produce the global optimum if the four properties are true in the general case. 
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Fig. 6 — Design procedure. 

ing constraint is violated and increase the accumulated cost of added 
positions appropriately. If the cost of the additional positions is less 
than the accumulated cost of the servers that have been removed, we 
determine if this solution is feasible. If it is feasible, we proceed as in 
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(i). If it is not feasible, either (ii) or (Hi) must be true. If the cost of 
the positions is more than the cost of servers, then no feasible solution 
with lower cost can be obtained with this number of servers and, hence, 
we reduce the number of servers by one and calculate the system char- 
acteristics with the new set of system parameters. Then either (i), 
(ii), or (Hi) must be true, and the appropriate action is then taken. 

In summary, the procedure removes servers until either the delay 
constraint is violated, in which case it terminates, or a blocking con- 
straint is violated. If a blocking constraint is violated, waiting posi- 
tions are added, and an additional server may be removed, depending 
on the incremental costs and on the feasibility of the tentative solution. 

A justification of the procedure is in order. We first discuss the case 
in which at least one blocking constraint has been violated. To reduce 
the blocking probability for each queue whose constraint has been 
violated, a position must be added to this queue (by Property 4) . The 
only other way to reduce the blocking probability is to add a server, 
but this branch has already been terminated. Assume that NS servers 
have been removed since the last "optimal" feasible solution and that 
the cost of all the positions added since the last "optimal" feasible 
solution is G. If G > NS X C, then the new parameters cannot give 
a lower cost solution and, by Property 4, no lower cost solution for 
this M exists. Therefore, we terminate the branch with M servers and 
begin the search of the branch with M — 1 servers (increment NS by 
1) with the present number of positions. At least this number of posi- 
tions must be considered, since, by Property 3, the reduction of M 
results in an increase in the blocking probabilities. If G S NS X C, 
this set of parameters may be a lower-cost solution. Therefore, we 
determine the system characteristics and see if the solution is feasible. 

We now prove that the procedure converges in a finite number of 
steps to a global optimum. 

Lemma 1: If a feasible solution for a given M has been found, the branch 
corresponding to that M can be terminated. 

Proof: Trivially, any further feasible solutions with that M must be 
more expensive, since these solutions must have more waiting positions. 

Q.E.D. 

Theorem 1 : The algorithm terminates in a finite number of steps. 

Proof: First, we know that an initial feasible solution can be obtained 
in at most 



[£w»-m w J 
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steps, as discussed earlier, where [V] + = max (0, x). Second, for any 
given M, the corresponding branch of the solution tree will be ter- 
minated in a finite number of steps. That is, since C is finite and C,- is 
nonzero for all i, then in a finite number of steps we will either find a 
feasible solution for this M, reach the point where the cost of the posi- 
tions added for this M exceeds C, or violate the delay constraint. In 
the first case, by Lemma 1, we know that we can terminate this 
branch. In the second case, since none of the positions can be removed 
and feasibility be maintained, no feasible solution of lower cost exists 
for this M . In the latter case, the procedure terminates. The number of 
iterations performed for a given M is bounded by the number of times 
positions are added before the cost of these additions exceeds C. 

Finally, since the maximum number of servers that need be con- 
sidered is finite, we reach the case in which the delay constraint is 
violated in a finite number of steps (at most, M m values of M). 

Since there are only a finite number of values of M to be considered 
and since, for each M, only a finite number of steps are performed, the 
algorithm terminates in a finite number of iterations. Q.E.D. 

Theorem 2: The solution (N*, M*, Z*) obtained upon termination of the 
algorithm is a global optimum. 

Proof: Assume that another configuration (N, M, Z) exists, such that 
all constraints of (I) are satisfied and Z < Z*. First, consider the case 
in which M > M*. By construction, the branch corresponding to M 
must have been searched and, as indicated in Theorem 1, the branch 
would have been terminated in a finite number of steps. If this branch 
had produced a feasible solution with a lower cost, it would have been 
retained. Hence, this case is not possible. 

Second, consider the case in which M < M*. From the algorithm, 
we know that the termination of the procedure implies that the delay 
constraint has been violated. We therefore know that either the branch 
with M produced a feasible solution with a cost larger than Z* (or else 
it would have been retained), or M is smaller than the value of M 
when the procedure terminates. If the latter is true, then M cannot 
produce a feasible solution since the delay and blocking probability 
for (N*, M) must be greater than those for (N*, M*) by Property 3 ; 
and, to reduce this delay, positions would have to be removed that 
would result in at least one blocking constraint being violated. Con- 
sequently, this case is a contradiction and, hence, (N*, M*, Z*) is the 
optimum. Q.E.D. 

One point should be noted : For the higher levels of blocking (>0.05), 
the solution (N (0) , M {0) ) is generally a feasible solution. However, as 
a result of the reduction in offered load to the servers because of the 
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blocking, M (0 ) can be reduced before any constraints are violated. This 
is of importance since, in practice, such systems have been engineered 
in such a manner that the configuration is (N (0) , M w ), which is ob- 
tained from use of the Erlang B and Erlang C formulas, as described 
earlier. 
A more concise mathematical summary of the algorithm follows. 

Optimization Algorithm: 

Step 1 : Initial Feasible Solution 

(i) Determine N$ m (i = 1, 2, • • •, I) from the Erlang loss formula 

where A^ 0) = min [n\B(n, a,-) ^ &,}. 
(it) Determine M w from the Erlang delay formula where 

ilf(o) = min {m\W(m, a) ^ d}. Let k = 1. 

kth. iteration : 

(Hi) Calculate £;(N(°\ Af<*_i )j a) and 5(N<°\ itf ( *-i), a). If the 
set of constraints (21) are satisfied, let M m = M ik -i)- The 
initial "optimal" feasible solution is N* = N<°>, M* = M<°>, 
and Z* = £U, CtN* + CM*. Set j = 1, iiV (0 > = N*, and go 
to Step 2. 

If at least one constraint of (21) is not satisfied, set M^> 
= M ( k-\) + 1, increment k, and return to (iii). 

Step 2: Solution Improvement 

In this step, the superscript j refers to the jth "optimal" value of 
M ; for a given value of j, the subscript r refers to the rth value of iV,-. 

jth. iteration : 

(i) NS = 1, (?«> = 0, r = 1. 
(ii) Reduce number of servers by one, M U) = M U ~ Y) — 1. If 

M<*> = 0, go to Step 3. 
(iii) Calculate .B^N^, ilf «>, a) and D(JfH*- l \ M<», a). 

(a) If D( r W*- l \ jfw», a) > d, go to Step 3. 

(b) If (21) and (22) are satisfied and if G U) ^ NS X C, then 
store (flftf-u, M 0) ) as the new "optimal" solution. That 
is, set N* = r N^\ M* = M™ and 

Z* = £ d rN {i - l) + CM™. 
»-i 

Set iN (i) = r N (>_1) , increment i, and go to (i). 
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(c) If (21) and (22) are satisfied but G ( ' -) > NS X C, then 
reduce the number of servers by subtracting 1 from M (i) , 
increment NS, and return to (Hi) . 

(d) If at least one blocking constraint of (21) is violated, i.e., 
S« = {*|fli( f N«-», M ( », a) > &<} is not empty, then add 
£«€*«i Ci to (?">. Let 

and 

If 6 <J) > NS X C, then decrement M (i) by 1, increment 
r, and go to (m). If G {j) ^ iVS X C, increment r, and go 
to (Hi). 

Step 3: Termination 

If D(JX li - l \ M U) , a) > d, then stop the procedure. The global 
optimum is (N*, M*, Z*). 

4.3 Numerical example 

To illustrate the algorithm, we examine a simple two-queue system 
that represents an automatic call distributor system used for credit 
checking. The company has subscribed to two Inward wats bands 
with a cost per trunk of $800 and $500. The two trunk groups each 
receive 15 erlangs of traffic in the busy hour. Calls, on the average, 
are 45 seconds in duration. The subscriber has requested a 5-second 
speed of answer and blocking objectives of P. 10 and P.05, respectively. 
The monthly cost of an attendant is $750. With these parameters, 
we begin the algorithm by using the Erlang loss formula with 15 
erlangs and the delay formula with 30 erlangs to obtain (N (0) , M (0 )), 
which are (18, 20; 34). As shown in Table I, this initial solution is feas- 
ible and hence will be stored as our tentative optimal solution, 
(N<°>, M<°>). 

We proceed to Step 2 of the algorithm in an attempt to improve the 
initial feasible solution. By decreasing M <0) by one, M a) = M m — 1, 
we obtain the system parameters (18, 20; 33) and the system char- 
acteristics (0.0912, 0.0504; 0.404), which indicate that this is not a 
feasible solution. Since the blocking constraint for trunk group 2 is 
violated, a trunk must be added to this group at a cost of $500. The 
accumulated cost of the additional trunks, $500, is less than the ac- 
cumulated cost of the removed attendants, $750. Therefore, we 
proceed by obtaining the system characteristics for this set of param- 
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Table 1 — An 


example of the optimization procedure 






Ni 


N, 


M 


Bi 


B, 


D 


Z($) 


Initial Feasible 


















Solution 


1 


18 


20 


34 


0.0887 


0.0481 


0.201 


49900. 






f2 


18 


20 


33 


0.0912 


0.0504 


0.404 


49150. 






3 


18 


21 


33 


0.0926 


0.0373 


0.546 


49650. 


Solution 

Improvement 




4 


18 


21 


32 


0.0970 


0.0409 


0.945 


48900. 


- 


5 


18 


21 


31 


0.1034 


0.0460 


1.550 


48150. 




6 


19 


21 


30 


0.0939 


0.0573 


2.990 


48200. 




* 


7 


19 


22 


30 


0.0976 


0.0466 


3.422 


48700. 






8 


19 


22 


29 


0.1120 


0.0572 


4.976 


47950. 


Termination 


9 


20 


23 


28 


0.1220 


0.0676 


8.864 


48500. 



* Optimal solution. 
System Parameters : 






ai = 15 erlangs, 
as = 15 erlangs, 
HT = 45 s, 


6i = 0.10, 
bi = 0.05, 

d = 5 s, 


ci = $800 
d = $500 
C = $750. 



eters, i.e., (18, 21 ; 33). As Table I indicates, this is a feasible solution, 
is a cost improvement, and hence is stored as the tentative optimum. 

As shown in Table I, the procedure continues from this point until 
(20, 23; 28), at which point the delay constraint is violated. The 
optimal solution is (19, 22; 30) at a cost of $48,700. We should note 
that the parameters (20, 23 ; 29) were not examined since the accumu- 
lated cost of added trunks, $1300, was greater than the accumulated 
cost of removed attendants, $750. 

The above solution is the global optimum for this constrained 
problem. However, practitioners might suggest that (18, 21; 31) is a 
more realistic design since, in fact, the blocking constraint is "es- 
sentially" satisfied (0.1034 vs 0.1000). This can be incorporated in the 
design procedure by allowing any solution that is within "e" of a 
blocking objective to be retained. The algorithm can then be applied 
as before. 



V. SUMMARY 

In this paper, an analysis of a particular multiserver, multiqueue 
service system has been presented. Examples of this type of system 
are the directory assistance systems used in the telephone companies 
and credit verification bureaus used by the credit-card industry, which 
use automatic call distributors. Expressions were derived for the 
equilibrium-state probabilities, and four properties of the system char- 
acteristics, overall average delay, and the blocking probabilities for each 
queue were given. 

These results were used in developing a procedure to obtain a least- 
cost system configuration to satisfy a given set of single-hour grade-of- 
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service constraints. That is, the procedure determines the number of 
waiting positions for each queue and the number of servers required to 
satisfy constraints placed on blocking probabilities and average delay 
at minimum cost. The work reported here should form the basis for 
the development of a practical method of traffic engineering and 
administration for small automatic call distributor systems. 
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